Film Grain Management (FGM, also referred to as Film Grain Technology, or FGT) has been presented as a new tool that allows encoding the grain in motion picture film by means of a parameterized model to be transmitted as parallel information for use by a video decoder. To support FGM, the Fidelity Range Extension (FRExt) Amendment to the ITU-T Rec. H.264|ISO/IEC 14496-10|MPEG-4 AVC|Joint Video Team (JVT) standard (hereinafter the “H.264 standard”) has defined a Film Grain Supplemental Enhancement Information (SEI) message. The SEI message describes the film grain characteristics regarding attributes such as size and intensity, and allows a video decoder to simulate the film grain look onto the decoded picture. The H.264 standard specifies which parameters are present in the film grain SEI message, how to interpret the parameters, and the syntax for encoding the SEI message in binary format. However, the H.264 standard does not specify the exact procedure to simulate film grain upon reception of the film grain SEI message by a video decoder. It is to be appreciated that FGM can be used jointly with any other video coding method, since FGM utilizes parallel information, transmitted from an encoder, that does not affect the decoding process.
In FGM, the encoder models the film grain of the video sequence and the decoder simulates the film grain according to the received information. The encoder can use FGM to enhance the quality of the compressed video when there is difficulty retaining the film grain. Additionally, the encoder has the option of removing or attenuating the film grain prior to encoding in order to reduce the bit-rate.
Film grain simulation aims at synthesizing film grain samples that simulate the look of original film content. Unlike film grain modeling, which is entirely performed at the encoder, film grain simulation is performed at the decoder. Film grain simulation is done after decoding the video stream and prior to display. Images with added film grain are never used within the decoding process. Being a post-processing method, synthesis of simulated film grain on the decoded images for the display process is not specified in the H.264 standard. The film grain simulation process includes the decoding of film grain supplemental information, transmitted in a film grain SEI message as specified by the Fidelity Range Extensions Amendment of the H.264 standard mentioned above.
In a previously disclosed prior art approach to film grain simulation, a set of specifications was disclosed to allow bit-accurate film grain simulation during normal playback. In order to support bit-accuracy with trick mode play (e.g. fast forward, reverse playback, jump to chapters, and so forth) an addendum to this first prior art approach (the addendum hereinafter referred to as the second prior art approach) was developed. In the second prior art approach to film grain simulation, bit-accuracy was achieved by transmitting the film grain SEI messages only preceding I frames and forcing the transmitted film grain SEI messages to be applied in decoding order. The second prior art approach ensures consistent film grain simulation for all the frames in normal playback as well as in trick mode play, with a minimum overhead in the video bit-stream due to the transmission of the film grain SEI messages. However, since the H.264 standard specifies that SEI messages are to be applied in display order (versus decoding order as specified in the second prior art approach), the solution proposed in the second prior art approach is not compliant with the H.264 standard. While this fact does not affect the perceived visual quality, it may prevent the deployment of the specifications disclosed in the second prior art approach in those forums where conformance to the H.264 standard is required.
Turning to FIG. 1, a film grain simulation in normal playback is indicated generally by the reference numeral 100. In particular, FIG. 1 shows the differences between film grain simulation in decode order 110 according to the second prior art approach and film grain simulation in display order 120 according to the H.264 standard. In this example, film grain SEI messages are sent preceding each I picture. Bold typeface denotes a picture where a film grain SEI has been inserted. In FIG. 1, decode order, the film grain SEI message sent with picture I2 is used in all following pictures until picture B10 is reached (inclusive). Horizontal lines above (decode order) or below (display order) pictures denote the film grain parameters (FG n) used with the pictures; for example, in FIG. 1, display order, film grain parameters FG 1 are used from the first I2 picture until the second B1 picture (inclusive). If SEI messages are assumed to apply to all frames following an I picture in decoding order, as specified in the second prior art approach, the film grain SEI message sent in I2 will apply to frames B0 and B1. However, if SEI messages are assumed to apply to all frames following an I picture in display order, as specified in the H.264 standard, frames B0 and B1 will be affected by the film grain SEI message of the previous I picture.
Accordingly, it would be desirable and highly advantageous to have a method for inserting film grain SEI messages in a video system in bit-accurate manner and in compliance with the H.264 standard.